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Over  the  Dost  ten  yearo,  research  In  the  field  of  dynamic 
programming  has  assumed  many  different  forms.  Sometimes,  the 
emphasis  has  been  upon  Questions  of  formulation  in  analytic 

terms  and  concepts,  sometimes  upon  the  problems  of  existence 
and  uniqueness  of  solutions  of  the  functional  equations  derived 
from  the  underlying  processes,  occasionally  upon  the  actual 
analytic  structure  of  the  solutions  of  these  equations,  some¬ 
times  upon  the  computational  aspects;  and  sometimes  upon  the 
applications — to  control  processes,  to  trajectories  of  various 
types,  to  operations  research,  tc  mathematical  economics. 

Inevitably,  the  result  of  this  quasi-ergodlc  behavior  has 
been  tc  ignore  a  number  of  significant  problems,  and  to  treat 
a  number  of  others  In  cavalier  fashion.  In  this  exposition, 
we  wish  to  focus  attention  upon  a  number  of  interesting, 
difficult,  and  significant  questions  in  analysis  which  arise 
naturally  out  of  the  functional  equation  technique  of  dynamic 
programming.  Our  aim  la  to  show  that  this  theory  constitutes 
a  natural  extension  of  classical  Investigations  and  that  the 
corresponding  problems  are  natural  gene rallzat Iona  of  problems 
of  classical  analysis. 
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DYNAMIC  PROGRAMMING  AND  CLASSICAL  ANALYSIS 

Richard  Bellman 


1.  Introduction 

Over  the  last  ten  years,  research  in  the  field  of  dynamic 
programming  has  assumed  many  different  forms.  Sometimes,  the 
emphasis  has  been  upon  questions  of  formulation  in  analytic 
terms  and  concepts,  [1,2];  sometimes  upon  the  problems  of 
existence  and  uniqueness  of  solutions  of  the  functional 
equations  derived  from  the  underlying  processes,  [2,5,4]; 
occasionally  upon  the  actual  analytic  structure  of  the  solu¬ 
tions  of  these  equations,  [5, 6, 7, 8, 9];  sometimes  upon  the 
computational  aspects,  [lO,ll];  and  sometimes  upon  the  appli¬ 
cations;  to  control  processes,  [2,12,15];  to  trajectories  of 
various  types,  [15,14]  ;  to  operations  research,  [lp, 16,17];  to 
mathematical  economics,  [18,19]  • 

Inevitably,  the  result  of  this  quasi— ergodlc  behavior  has 
been  to  Ignore  a  number  of  significant  problems,  and  to  treat 
a  number  of  ethers  In  cavalier  fashion.  In  this  exposition,  we 
wl3h  to  focus  attention  upon  a  number  of  Interesting,  difficult, 
and  significant  questions  in  analysis  which  arise  naturally  out 
of  the  functional  equation  technique  of  dynamic  programming. 

Our  aim  is  to  show  that  this  theory  constitutes  a  natural 
extension  of  classical,  investigations  and  that  the  correspond¬ 
ing  problems  are  natural  generalizations  of  problems  of 
classical  analysis. 

As  always  in  studying  new  areas,  there  is  the  hope  that 
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light  throw]',  lr,  VhU  v-1'.v.ln  territory  will  bo  ref iec tad  back 
upon  still  hidden  parts  .f  t ’ ».-s  cIcihhVc-jI  domain. 

We  shall  restrict  our  attention  here  to  deterministic 
processes,  reserving  for  other  times  any  discussion  of  more 
complex  and  arcane  processes  arising  fi-ora  the  study  of 
stochastic  and  adaptive  processes,  [2] .  These  new  areas  of 
analysis  where  many  different  sub— disciplines  such  as  algebra, 
topology,  differential  equations  and  probability  theory  merge 
and  lose  their  separate  identity  in  f'll-embraclng  problems 
offer  bountiful  and  boundless  regions  for  research.  The  reader 
familiar  with  the  concepts  and  problems  discussed  here  can 
readily  construct  for  himself  corresponding  questions  involving 
uncertainty . 


?.  Mnl  ti.~t:  Dec  1*4 on  PrccesHes 

Pyruir.. r  ;  ro^i^nm  ing  is  «  mathematical  theory  of  multistage 

decls4  .r.  prccer.sea.  Txj uOttona  «>r  tM.  theory  may  be  P' und  in 
[l]  and  [?J  .  Th  t  r  v  le:r m  .  w  will  he  very  much  n  :v 

neartir.yfui  wl4;:  ir,  the  ccrt>»xt  of  dynamic  programming.  The 
rerder  whc  wishes,  i  owever,  may  \  -'jr  .  i  e  h1‘  insertions  of  moti¬ 
vation  and  n’Jiir.l  the  p  ir  blem?  *hr.t  follow  a?  n  ni’cdrw  pulled 
cut  ^  the  bj;?. 

j.  The  Calculus  of  Variations 

Perhaps  the  most  interesting  and  important  example  of  a 
multistage  lecislon  process  of  continuous  type  is  the  calculus 
of  variations.  Consider  the  scalar  variational  problem  of 
maximizing  the  integral 


(1) 


J(y)  -,/T  Hx.yjdt, 

u0 
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over  all  functions  y,  where  x  and  y  are  related  by  means 
of  the  dl  f  fpr**nM  «^’j.itlon 

(2)  St  '  0!’'’y)'  X'0)  ’  =• 

Introducing  the  function 


(M 


f(c,T'  -  Max  J'y), 


we  obtain  from  the  principle  of  optimality,  [lj  ,  the  nonlinear 
partial  dl  f ferrtnt  1  ?.?  oq’ntior. 


(4) 


jijr  •  Max  P(c,v)  ♦  G  ( c ,  v  ) ji- 


with  the  lnitla^  condition 


J 


f  {  C  ,0  y  *  0  . 


See  [l  ,2]  . 

We  oh all  uso  tula  equation  as  a  pivot  for  some  of  tno 
subsequent  problems  we  ona*i  preoent. 


4.  Existence  and  Uniqueness 

The  equation  in  ,3.4)  Is  derived  in  a  formal  fashion, 
much  as  the  Euler  equation  la  customarily  obtained.  The  first 
problem  wo  pose  la  that  of  finding  conditions  upon  the 
functions  P(x,y)  and  Q(x,y )  which  ensure  the  existence  and 
uniqueness  of  a  solution  of  tno  unconventional  nonlinear  partial 


differential  equation  of  (3.4). 
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One  way  to  approach  this  problem  is  to  employ  the  reeulte 
of  the  classical  calculus  of  variations.  These  enable  us  to 
establish  the  existence  of  a  solution  of  the  original  variation¬ 
al  problem,  under  certain  rather  rigid  conditions  upon  P(x,y) 
and  0(x,y). 

This  is  not  a  particularly  satisfactory  procedure  for  a 
number  of  reasons.  In  the  first  place,  we  would  like  to  use 
Equation  (3.4)  to  resolve  the  original  variational  problem. 
Secondly,  we  want  to  use  analogues  of  (3.4)  to  study  variational 
problems  which  partially  or  wholly  escape  the  classical  theory. 
This  will  be  the  substance  of  the  following  section. 

3.  Constraints 

A  question  of  much  analytic  interest,  and  one  which  in  any 
case  is  rudely  thrust  upon  us  by  a  great  number  of  feedback 
control  and  trajectory  processes,  is  that  of  solving  the 
variational  problem  of  (3.1)  and  (3-2)  under  the  additional 
constraint 

(i)  |y(t;|  <  k,  o  ^  t  <  t. 

It  is  certainly  surprising  that  a  simple  condition  of  this 
nature  should  so  effectively  stymie  the  usual  approach,  but 
nonetheless,  true.  For  a  detailed  discussion,  see  [2,20]  . 

As  a  result  of  the  presence  of  the  foregoing  restriction, 
the  equation  of  (3.4)  is  replaced  by  the  equation 
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<2'  [P(C’V)  +  0(c'v)39- 

One  might  surmise  that  the  presence  of  the  constraint  would 
simplify  the  investigation  of  existence  and  uniqueness  of 
solution,  as  it  certainly  does  the  computational  solution  of 
equations  of  this  nature.  However,  equations  of  the  foregoing 
type  have  not  been  rigorously  investigated  as  yet. 

6.  Discrete  Version — I 

A  standard  route  to  the  goal  of  existence  of  a  solution  of 
a  functional  equation  Involving  derivatives  is  uy  way  of 
difference  equations.  This  approach  is  of  the  utmost 
importance  in  connection  with  the  numerical  solution  of  these 

equat  ior.L  . 

Let  us  then  in  place  of  (3*4)  write  the  recurrence  relation 

(l'  ~  -f. m  Max  r?(c,v)  ♦  .f Cc"6fTl 

2b  v  L  d0 

where  c  and  T  assume  respectively  values  of  the  form  +  kS, 
lb.  Kb  b  — *  0,  6  — ♦  0,  this  relation  formally  approaches 
that  of  (3.4). 

The  problem  now  is  that  of  determining  the  connection  be¬ 
tween  the  solution  of  (1)  and  the  possible  solution  of  (3.4), 
first  under  the  assumption  that  (3.4)  has  a  solution  and 
secondly  in  the  hope  of  using  the  solution  of  (1)  to  establish 
the  existence  of  a  solution  of  (3.4). 

Closely  associated  with  questions  of  this  nature  is  the 
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question  of  numerical  stability  of  a  computational  algorithm 
such  as  (1).  Finally,  let  us  note  that  this  is  only  one  of  a 
large  set  of  possible  discrete  approximations  to  (3.4). 

7.  Discrete  Version — II 

To  obtain  a  discrete  version  of  a  different  type,  we  use 
an  idea  of  some  importance  as  far  as  approximation  techniques 
are  concerned.  In  place  of  approximating  to  the  exact  equation 
describing  the  process,  we  can  employ  the  exact  equation  for  an 
approximating  process. 

Hence,  in  place  of  the  continuous  decision  process 
described  by  the  equations  of  (4.1)  and  (4.2),  let  us  consider 
the  problem  of  determining  the  sequence  jy^  which 
minimizes  the  function 


N-l 

(1) 

JK(y) 

*  JJ'tvy-JA. 

k-0 

where 

x,,  and 

A 

are  related  by  means  of  the  equation 

(2) 

xk+l  “ 

'  \  ’  a'-\’yK'A’  *0  =  °>  *  "  0<1'2’  ••• 

Here 

< 

X 

1 

X 

.  yk  -  y (xfl)  .  n &  -  t. 

Setting 


(3)  fN(c)  -  Min  JN(y ), 

y 

we  obtain  the  recurrence  relation 

(4)  f„(c)  -  Max 

v 


F(c,v)&  +  fN_i(c  ♦  . 
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with 

(5  )  f0(c )  -  Max  F(c,v)A. 

J  v 

Recurrence  relations  of  this  nature  have  been  quite 
successful  in  the  obtaining  of  computational  solutions;  see 
[10,14,21]  . 

Observe  that  as  A  — +  0,  the  equation  in  (4)  formally 
reduces  to  that  of  (5*4).  Yet,  unlike  (5.1),  it  involves  only 
one  small  quantity  A,  and  no  ratio  of  small  quantities  such 
as  A/6.  T3iis  la  a  most  important  point  in  connection  with 
numerical  stability. 

A  small  amount  of  work  has  be^n  done  on  the  subject  of  the 
rorve^hnce  of  the  solution  of  (4)  to  the  solution  of  (3.4); 
see  [22, 1,23# 24]  .  Much  remains  to  be  done. 

8.  An  Application 

We  have  mentioned  above  the  possibility  of  applying  ideas 
and  techniques  developed  in  the  new  field  of  multistage 
decision  processes  to  classical  equations.  Let  us  give  an 
example  of  this . 

Consider  the  partial  differential  equation 
(1  )  ut  -  uux>  u ( x , 0 }  -  g(x) , 

an  equation  which  pussesces  a  shock  discontinuity.  In  place  of 
the  usual  finite  difference  scheme,  a  la  (6.1),  let  us  borrow 
the  approach  of  (7.4^  and  use  the  relation 
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(2)  u(x,t  ♦  A)  -  u(u  ♦  u(x,t'iA,t), 

as  an  approximation  to  (1). 

Excellent  results  have  been  obtained,  even  In  the 
Immediate  neighborhood  of  the  shock,  using  this  algorithm;  see 
[253*  No  work  has  been  done,  however,  on  the  question  of  tne 
convergence  of  the  solution  of  (2)  to  the  solution  of  (1)  as 
&  — *  0,  for  this  equation  or  for  more  general  equations.  For 
another  application  of  this  Idea,  see  [26]  . 

9.  Approximation  In  Policy  Space 

Another  attack  on  the  basic  problem  of  establlsning 
existence  and  uniqueness  of  solutions  of  the  equation  In  (4.7, 
or  (o.2)  is  by  means  of  the  method  of  successive  approximations. 
In  place  of  the  usual  approach,  let  us  Invoke  the  technique  of 
approximation  In  policy  space ;  see  [\ , 2]  . 

We  begin  by  guessing  an  Initial  policy ,  v^  »  v^(c,T), 
and  using  this  policy  to  determine  a  return  function 
fg  •  fQ(c,T),  by  means  of  the  linear  partial  differential 
equation 

(1)  yr  "  F(c*V  *  0(c'vo,?cfi '  fo(c'0;  "  °* 

Having  determined  fQ(c,T',  let  us  determine  the  function 
v^  by  the  condition  that  It  maximize  the  function 
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Using  this  new  policy— function,  let  us  determine  the  new  return 
function  f1  by  way  of  the  linear  partial  differential  equation 
b  T 

(3)  ■  P(c,v^)  ♦  Ofc.v^^l,  f1(c,0'i  •  0. 


Continuing  in  this  fashion,  we  obtain  a  sequence  of  functions 
^  f ^  |  and  iv^J.  In  view  of  the  manner  in  which  is  deter¬ 

mined,  it  is  clear  that 


3f0 

W 


F(c ,vQ) 


afo 

*  Q(c»v0^^c~ 


2f, 


<  F(c,v1  j  + 


Prom  this,  we  would  expect  to  find  that  fQ(c,T)  <  f1(c,T)  for 
T  >  0,  and  this  is  actually  the  case.  Oenerally,  the  merit  of 
this  approximation  method  is  that  it  yields  monotonic  behavior, 


(3)  f0(c,T)  <  f1(c,T;  <  •••  <  fn(c,T)  <  •••. 

important  problem  is  to  determine  conditions  under 
which  this  sequence  converges  to  a  solution  of  (.5.4},  and,  of 
course,  to  investigate  th*  application  of  this  technique  to 
more  general  equations.  Some  preliminary  results  for  a  much 
simpler  problem  may  be  found  in  [l]  ,  Chapter  11. 


10.  Positive  Operators 

A  crucial  result  in  the  foregoing  procedure  is  the  con¬ 
clusion  f1  >  fg  for  T  >  0  as  a  consequence  of  the  equality 
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and  the  inequality 
3f 

(2  J  » vi  ^  ♦  G(c»vljJc^' 

granted  common  initial  conditions  at  T  «  0. 

Questions  of  this  nature  are  part  of  the  theory  of  positive 
operators.  Oiven  a  relation  Au  2  0»  where  A  is  an  operator, 
we  wish  to  determine  when  tnts  implies  u  >  0.  Thi)  type  of 
investigation  was  initiate!  by  Caplygin,  [27];  see  [28,29,30j 
for  further  results  and  references .  Again  much  remains  to  be 
done  In  this  field. 

11.  Quaslllnearlzat ion 

Let  us  now  indie  it*  another  aprl  ‘ oat  i<  >n  of  new  technlpj'*?' 
developed  in  the  theory  of  dynamic  programming  to  classical 
analysis.  We  have  Just  seen,  in  ^9 ,  that  equations  of  the 
special  form  of  (3.4  1  can  be  approached  along  a  route  which  is 
not  open  to  the  equations  of  classical  analysis.  In  view  of 
the  raonotonlclty  of  approx.LT.ctlo0,  1  '»«?ry  '»aiu:-*bie  analytic  and 
computational  aid,  it  may  be  worth  devoting  some  effort  t  the 
question  of  converting  an  equation  of  conventional  type  into 
one  of  the  form  of  (3.4). 

What  wti  are  doln*  Is  transforming  an  equation  arising  from 
a  descriptive  process  into  one  which  arises  from  a  variational 
process  Tills,  of  course,  is  a  familiar  Idea  in  analysis  and 
one  of  great  power  and  versatility.  The  way  in  which  we  do  it 
is,  however,  new. 

To  give  a  simple  example  of  the  technique  which  can  be 
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used,  consider  the  Rlccatl  equation 

(1)  u*  -  u2  +  a(t) ,  u  (0)  -  c  . 

Write 

(2)  u2  -  Max  (2uv  -  v2), 

v 

so  that  (1)  assumes  the  form 

(3)  u'  -  Max  |^2uv  -  v2  ♦  a(t)J  ,  u(0)  ■  e. 

Consider  the  related  linear  equation 
(4  ;  IT'  -  2Uv  -  v2  +  a(t ),  U(C)  -  c, 

whose  solution  we  write  In  the  form 
(5)  U  -  T(v.t). 

We  suspect  that 

(5)  u  -  Max  U  •  Max  T(v,t), 

v  v 

a  fact  which  la  a  consequence  of  the  positivity  of  the  operator 
d/dt  -  2v,  In  the  sense  of  §10.  Since  we  can  obtain  an 
explicit  representation  for  T(v#t)  In  terms  of  Integral 
equations,  (o)  furnishes  an  Interesting  representation  for  the 
solution  of  ( 1 ) . 

Oenerally,  If  f(u,t)  Is  convex  as  a  function  of  u  for 
0  <  t  <  tQ#  we  can  write 


(7) 


P-1 304 
9-28-59 
1? 


f(u,t)  -  Max  [f(vfti  ♦  (u  -  vjfy(vft)J, 

and  if  f  (u,t )  is  concave,  we  can  write 

(8)  f(u,t)  -  Min  ff(v,t)  4  (u  -  v)f  (v,t)  . 

v 

'Riese  representations  enable  us  to  treat  differential  equations 
of  the  form 

(9)  u'  -  f *u,t  ) , 

and,  more  generally,  to  transform  functional  equations  of  the 
form 

(10)  Au  -  f  (u,t  ) , 

where  A  la  an  operator,  into  quasllinear  equations  of  the 

forra 

(11;  Au  -  Max 

v 

Having  obtained  this  form,  we  can  employ  approximation  in 
policy  space,  and  proceed  as  above.  This  representation  also 
has  important  computational  advantages.  For  a  more  detailed 
discussion,  with  many  examples,  see  [j?B,29j- 

12.  Semi— Qroup  Theory 

The  modem  theory  of  semi— groups  of  transformations,  [^l}, 
deals  with  functional  equations  of  the  form 


: 


f  (  V  ,  t  )  4  (  U  -  V  i  f 


(  V  ,  t  ) j 


T 
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(1)  3T  "  Au' 

where  A  Is  a  linear  operator.  A  particular  equation  of  thla 
type  is  the  linear  partial  differential  equation  of  (9.1).  The 
functional  equations  associated  with  dynamic  programming  pro— 
ceases  of  continuous  type  have  the  form 

(2)  -  Max  [A(v)u  +  b(v)]  . 


In  view  of  tne  fact  that  (2)  contains  (1),  we  can  expect 
a  diffusion  of  knowledge  in  both  directions.  In  the  first  place, 
in  view  of  the  quaalllnearlty  of  (2),  we  may  expect  that  a  large 
part  of  sem.*,. -group  theory  can  be  applied  to  the  question  of  the 
existence  of  solutions  of  (2).  In  this  way  we  would  hope  to 
obtain  far  stronger  results  in  the  calculus  of  variations  than 
those  currently  available.  In  particular,  we  would  aspire  to  a 
more  complete  theory  for  variational  problems  with  constraints. 

Secondly,  we  may  expect  to  utilize  the  results  of  the 
linear  theory,  and  some  of  the  results  and  techniques  of 
dynamic  programming,  tc  obtain  a  theory  of  nonlinear  equations 
whicn  can  be  written  in  the  font,  of  (2). 


13.  Multidimensional  Variational  Problems 

Consider  tne  problem  of  maximizing  the  Integral- 


(1) 


J(u^  •Jpt  (u,ux,uy  )dA, 


where  the  integration  is  over  the  interior  of  a  two-dimensional 
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region  R,  and  the  value  of  u  Is  prescribed  on  the  boundary 
B  of  R, 

(2)  u  -  g(P),  P  *  B. 

In  a  procedure  completely  analogous  to  the  one— dimensional  case, 
we  may  introduce  the  functional 

(3)  f(g(P);R)  -  Max  J(u). 

u 


Considering  a  sequence  of  shrinking  regions,  it  Is  not 
difficult  to  obtain  a  functional  equation  for  f(g(P);R).  The 
derivatives  that  occur  will  now  be  functional  derivatives,  cf. 

Although  this  technique  ha3  been  used  to  obtain  the 
Hadamard  variational  formula  for  the  Green’s  function  of  a 
region,  and  other  results,  [35,54,53,36],  no  work  has  been  done 
on  the  existence  and  uniqueness  of  solutions  of  equations  of 
this  nature . 

An  Interesting  side  problem  associated  with  variational 
questions  of  this  nature  Is  that  of  determining  the  functional 
form  of  f(g(P);R).  Sven  for  the  classical  problem  involving 
the  Dirlchlet  functional, 


(») 


J(u) 


4 


u>A, 


deep  and  subtle  analysis  Is  required;  see  Osborn,  ([57],  where 
further  references  may  be  found. 
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14  .  Stability  and  Asymptotic  Behavior 

Aa  soon  as  the  fundamental  questions  of  existence  and 
uniqueness  of  solution  have  been  disposed  of,  we  can  turn  to 
the  deeper  and  more  Interesting  problems  of  the  analytic 
structure  of  the  solution.  In  particular,  we  would  like  to 
examine  the  stability  properties  of  the  solution  and  its 
asymptotic  behavior  as  t  -»  oo  .  A  small  amount  of  work  has 
been  done  in  this  direction,  but  no  systematic  theory  has  been 
constructed;  see  'i7,38,59]  . 

A  theory  of  this  nature  can  be  based  upon  an  extension  of 
the  present  theory  of  positive  operators;  see  [40,41,42]. 

15.  Implicit  Variational  Problems 

Sc  far  we  have  considered  variational  processes  of  fixed 
duration.  It  is  of  interest  to  consider  processes  in  which 
the  upper  limit  T  depends  upon  the  policy  used,  and  those  of 
still  more  general  nature  .  Those  are  examples  of  implicit 
variational  problems . 

One  of  the  most  important  examples  of  this  type  of  problem 
la  furnished  by  "bang— bang"  control.  Suppose  that  we  have  a 
system  S  ruled  by  a  vector  differential  equation 

(1)  -  g(*,y),  x(0)  -  c, 

where  the  components  of  y  are  subject  to  the  restrictions 

\&  /  l  y  ±  V  - )  1  js  ^  ^ ^  "  1  »  2,  . . .  ,N, 


or 
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(b)  y1(t)  -  +  k1,  1  - 

We  wish  to  choose  y  In  such  a  way  as  to  minimize  the  time 
required  to  force  tne  system  from  the  initial  state  c  to 
nother  state,  say  0. 

Por  the  linear  case, 

(3)  •  Ax  -f  y ,  x(Q)  -  c, 

a  threat  deal  has  brr*n  a^ne  U3 1  n _j  k  variety  >f  technl  :ju'*3 ;  see 
[43, ^4,4 for  further'  references. 

10.  Purt’ner  Directions 

It  is  easy  to  pose  a  number  of  additional  questions  of 
interest  if  we  admit  two  person  processes,  multistage  games; 
cf.  [4,46],  and  If  we  Introduce  stochastic  and  adaptive  pro¬ 
cesses  in  general;  [1, 2,39*47]  . 

The  range  of  investigation  Is  now  so  broad,  and  so 
different  in  many  ways  from  the  classical  tableau,  that  we 
feel  It  better  to  present  113  cuss  ions  of  this  nature  in 
separate  publications. 


P-1  ft  04 
9—26-69 

17 


RSRI-lRtfNC 


1.  P.  VI  Ir.fij  ,  JynamV  F'ix ri’anmln g ,  Princeton  Univ  .  Press, 

Prince ton ,  ri 7  77,  1 9**. 

2  .  - ,  Dyr m . .  1  r jgramml ng  and  Adaptive  Control  Pro— 

cesses ,  to  appear. 

3.  - ,  ”50106  functional  equations  In  the  theory  of 

dynamic  programming — I:  functions  of  points  and  point 
transformation,"  Trans,  Amer.  Math.  Soc . .  vol.  80,  1955# 

PP .  51—71. 

4 .  - ,  "Functional  equations  in  the  theory  of  dynamic 

programming — III,"  Rend.  Clr.  Mate.  Palermo,  ser.  2, 
tomo  5#  1958,  pp.  1-23 . 

6.  - ,  "A  variational  problem  with  constraints  In 

dynamic  programming,"  J.  Soc.  Ind,  Appl.  Math.,  vol.  4, 
1950,  pp.  48— ol .  '  ’  '  " 

o.  - ,  "On  a  class  of  variational  problems,"  Q.  Appl. 

Math.,  vol.  14,  1957,  pp.  353-359. 

7.  - ,  "Functional  equations  in  the  theory  of  dynamic 

programming — V :  positivity  and  quas  1-1  inear ity , "  Proc  . 

Hat.  Acad.  3c 1.  USA,  vol.  4l,  1955,  PP .  74>-748. 

8.  R.  Bellman,  I.  Ollcksberg,  and  0.  Gross,  "On  the  optimal 

Inventory  equation,"  Management  Scl.,  vol.  2,  1955, 
pp .  8>-104  . 

9.  R.  Bellman,  "On  some  applications  of  dynamic  programming  to 

matrix  theory,"  Illinois  J.  Math.,  vol.  1,  1957, 

PP.  297-501. 

10.  R.  Bellman  and  S.  Dreyfus,  Computational  Aspects  of  Dynamic 

Programming,  Princeton  Univ.  Press,  Princeton,  k.  j 
to  appear. 

11.  - ,  "A  bottleneck  situation  Involving 

Interdependent  Industries,"  Naval  Research  Log.  Q..  vol.  5, 

1958,  pp.  21-28. 

12.  R.  Bellman,  S.  Dreyfus,  and  R.  Kalaba,  "Dynamic  programming 

trajectories  and  space  travel,"  Proc .  Fourth  Symp .  on 
Astronautics ,  Los  Angeles,  1959#  to  appear. 

15-  R.  Bellman  and  S.  Dreyfus,  "An  application  of  dynamic  pro¬ 
gramming  to  the  determination  of  optimal  satellite 
trajectories,"  J.  British  Interplanetary  Soc.,  to  appear. 


P-1804 

9-28-69 

18 


14.  T.  Cartalno  and  S.  Dreyfus,  "Application  of  dynamic  pro¬ 

gramming  to  the  airplane  minimum  time— to— climb  problem," 
Aero,  Sng.  Re^.,  1 9L>7  • 

15.  S.  Dreyfus,  "a  note  on  an  Industrial  replacement  process, " 

Operational  Research  Q.,  December,  1957. 


10.  R.  Bellman,  "Equipment  replacement  policy,"  J.  Sue.  Ind . 
Appl .  Math. ,  vol.  3,  1955,  pp .  1 53-1 36. 

17.  R.  Howard,  Discrete  Dynamic  Programming  Processes ,  Thesis, 

Massachusetts  Institute  of  Technology ,  - . 

18.  R.  Beckwith,  Analytic  and  Computational  Aspects  of  Dynamic 

Programming  Processes  of  Ttlgh  Dimension,  Ph.D.  TheBla. 
Purdue  University]  1)59 . 

19.  R.  Bellman,  I.  Qlicksberg,  and  0.  Gross,  "On  the  optimal 

Inventory  equation,"  Management  Science,  vol.  2.  1955. 
pp .  83-104 . 

20.  R.  Bellman,  "Dynamic  programming  and  its  application  to  tt.e 

variational  problems  In  mathematical  economics,"  Proc . 
Symp .  In  Calc  il  *s  of  Variations  and  A^licatlona ,  Arne r . 
Math.  Soc7,  Chicago,  April,  i"$£b,  pp.  II5— published 
by  McGraw-Hill  Book  Co.,  Inc.,  New  York. 

21.  R.  Bellman  and  S.  Dreyfus,  "On  the  computational  solution 

of  dynamic  programming  processes — I:  on  a  tactical  air 
warfare  model  of  Mengel,"  J .  Oper .  Re3  . ,  vol.  o,  1956, 
pp .  o5-7 1 . 

22.  R.  Bellman,  "Functional  equations  in  the  theory  of  dynamic 

programming — VI:  a  direct  convergence  proof,'  Ann.  Math., 
vol.  03,  1957,  pp  .  215-225. 

23*  H.  Osoorn,  "The  problem  of  continuu  .u  programs,"  Pacific 
J.  Math.,  vol.  o,  1950,  pp.  721-/,51. 

24.  W.  Fleming,  Discrete  Approximations  toSeme  Continuous 

Dynamic  Programming  Processes,  The  RAND  Corporation, 
Research  Memurand  in  RM-15C1,  June  2,  195:'. 

25.  R.  Bellman,  I.  Cherry,  an!  0.  M.  Wing,  "A  note  on  the 

nomerlcal  Integration  of  a  class  of  nonlinear  hyperbolic 
equations,"  Q.  App 1 .  Math . ,  vol.  lo,  1958,  pp .  151—163. 

2o .  R.  Bellman,  "On  the  nonnegativity  of  solutions  of  the  heat 
equation,"  B  11.  Unlone  Mateiaatlco,  vol.  1?,  *h''T, 

;  :  .  529-  25- 


P-1804 

9-28-59 

19 


27.  S.  Caplygln,  A  Hew  Method  i‘or  the  Approximate  Integration 

of  Solution  of  Differential  Equations,  Princeton  unlv . 
?ress,  Prlnoeton,  ft.  J.,  to  appear;  translated  by 
R.  Bellman  and  R.  Kalaba. 

28.  R.  Bellman,  "Functional  equation*  In  the  theory  of  dynamic 

programming — V:  positivity  and  quail-linearity,"  Proc. 

Nat.  Acad.  Sci.  USA,  vol .  41,  1955,  pp.  743-746. 

29.  R.  Kalaba,  "On  nonlinear  differential  equations,  the  maxi¬ 

mum  operation  and  monotone  convergence,"  J.  Nath,  and 
Mech.,  vol.  6,  1959,  PP-  519-574. 

30.  E.  F.  Beckenbach  and  R.  Bellman,  Inequalities ,  vol.  1, 

Krgebnlsse  der  Math.,  to  appear. 

31.  B.  Hllle  and  R.  Phillips,  Functional  analysis  and  semi¬ 

group  s ,  Amer.  Math.  Soc .  tiolloq.  Publ.,  vol.  3i,  1958. 

32.  R.  Bellman,  "Dynamic  programming  and  a  new  formalism  in  the 

theory  of  integral  equations,  Proc.  Nat.  Acad.  Sci.  USA, 
vol.  41,  1955,  PP-  31-34. 

33*  R-  Bellman  and  H.  Osborn,  "Dynamic  programming  and  the 
variation  of  Oreen's  functions,"  J.  Math,  and  Mech., 
vol.  7,  1958,  pp.  81-86. 

34 .  R.  Bellman,  "Functional  equations  in  the  theory  of  dynamic 
programming — VIII:  the  variation  of  Oreen's  functions 
for  the  one-dimensional  case,"  Proc.  Nat.  Acad.  Sci.  USA, 
vol.  43,  1957,  pp.  839-841 .  ~ — 

35*  P  •  Bellman  and  S.  Lehman,  "Functional  equations  in  the  theory 
of  dynamic  programming — IX:  variational  analysis,  analytic 
continuation,  and  imbedding  of  operators,"  Proc.  Nat.  Acad. 
Sci.  USA,  vol.  44,  1958,  pp.  905-907. 

36.  - ,  "Functional  equations  in  the 

theory  of  dynamic  programming — X:  resolvents,  characteris¬ 
tic  functions  and  values,"  Duke  Math.  J..  to  appear. 

37.  H.  Osborn,  "The  Dlrlchlet  functional,"  J.  Math.  Analysis  and 

Appl . ,  to  appear. 

38.  R.  Bellman,  "A  Markovian  decision  process,"  J.  Math,  and 

Mech . ,  vol.  6,  1957,  pp.  679-t>84.  ~  — * 

39.  - ,  "Directions  of  Mathematical  research  in  non¬ 

linear  circuit  theory,"  PQCT  Trans . ,  to  appear. 

40.  M.  Kreln  and  M.  A.  Rutman,  Linear  Operators  Leaving 

Invariant  a  Cone  in  Banach  Space,  Amer.  Math.  Soc. 
Translation  No.  26,  1950. 


P-1804 

9-28-89 

20 


41.  T.  S.  Harris,  Branching  Processes,  Jtrgebnlsse  der  Math., 

to  appsar. 

42.  R.  Bellman,  Introduction  to  Matrix  Analysis,  McGraw-Hill 

Book  Co  . ,  Inc . ,  New  Vork .  19^9* 

4).  R.  Bellman,  1.  Glickaberg,  and  0.  Oross,  "On  the  'bang-bang' 
control  problem,"  Q.  Appl .  Math . ,  vol.  14,  19bb, 

pp.  11-18. 

44.  J.  P.  LaSalle,  "On  time  optimal  control  systems,"  Proc . 

Nat.  Acad.  Scl.  USA,  vol.  4?j,  1989,  pp .  87>677. 

4^.  R.  Bellman  and  J.  M.  Richardson,  "On  the  application  of 
dynamic  programming  to  a  class  of  implicit  variational 
problems,"  Q.  Appl .  Math . ,  to  appear. 

4b.  Studies  in  Qame  Theory—-!,  II,  III,  Annals  of  Math.  Studies, 
Princeton  University  Press,  Princeton,  N.  J. 

47.  R.  Bellman  and  R.  Kalaba,  "A  Mathematical  theory  of  adaptive 
control  processes,"  Proc.  Nat.  Acad.  Scl.  USA,  to  appear. 


